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Abstract 



A measure of complexity based on a probabilistic description of physical systems 
is proposed. This measure incorporates the main features of the intuitive notion of 
such a magnitude. It can be applied to many physical situations and to different 
descriptions of a given system. Moreover, the calculation of its value does not require 
a considerable computational effort in many cases of physical interest. 
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On the most basic grounds, an object, a procedure, or system is said to be 
"complex" when it does not match patterns regarded as simple. This sounds rather 
like an oxymoron but common knowledge tells us what is simple and complex: 
simplified systems or idealizations are always a starting point to solve scientific 
problems. The notion of "complexity" in physics [| starts by considering the 
perfect crystal and the isolated ideal gas as examples of simple models and therefore 
as systems with zero "complexity". Let us briefly recall their main characteristics 
with "order", "information" and "equilibrium". 

A perfect crystal is completely ordered and the atoms are arranged following 
stringent rules of symmetry. The probability distribution for the states accessible 
to the perfect crystal is centered around a prevailing state of perfect symmetry. A 
small piece of " information" is enough to describe the perfect crystal: the distances 
and the symmetries that define the elementary cell. The "information" stored in 
this system can be considered minimal. On the other hand, the isolated ideal gas is 
completely disordered. The system can be found in any of its accessible states with 
the same probability. All of them contribute in equal measure to the "information" 
stored in the ideal gas. It has therefore a maximum "information". These two 
simple systems are extrema in the scale of "order" and "information". It follows 
that the definition of "complexity" must not be made in terms of just "order" or 
"information" . 

It might seem reasonable to propose a measure of "complexity" by adopting some 
kind of distance from the equiprobable distribution of the accessible states of the 
system. Defined in this way, "disequilibrium" would give an idea of the probabilistic 
hierarchy of the system. "Disequilibrium" would be different from zero if there are 
privileged, or more probable, states among those accessible. But this would not 
work. Going back to the two examples we began with, it is readily seen that a 
perfect crystal is far from an equidistribution among the accessible states because 
one of them is totally prevailing, and so "disequilibrium" would be maximum. For 
the ideal gas, " disequilibrium" would be zero by construction. Therefore such a dis- 
tance or "disequilibrium" (a measure of a probabilistic hierarchy) cannot be directly 
associated with " complexity" . 

In figure 1. we sketch an intuitive qualitative behaviour for "information" H 
and "disequilibrium" D for systems ranging from the perfect crystal to the ideal 
gas. This graph suggests that the product of these two quantities could be used as 
a measure of "complexity": C = H ■ D (Fig. 1.). The function C has indeed the 
features and asyntotical properties that one would expect intuitively: it vanishes for 
the perfect crystal and for the isolated ideal gas, and it is different from zero for 
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the rest of the systems of particles. We will follow these guidelines to establish a 
quantitative measure of "complexity". 



Before attempting any further progress, however, we must recall that "com- 
plexity" cannot be measured univocally, because it depends on the nature of the 
description (which always involves a reductionist process) and on the scale of ob- 
servation. Let us take an example to illustrate this point. A computer chip can 
look very different at different scales. It is an entangled array of electronic elements 
at microscopic scale but only an ordered set of pins attached to a black box at a 
macroscopic scale. 

We shall now discuss a measure of "complexity" based on the statistical descrip- 
tion of systems. Let us assume that the system has N accessible states {x%, X2, xjv} 
when observed at a given scale. We will call this an N-system. Our understanding of 
the behaviour of this system determines the corresponding probabilities {pi,P2, ■■■,Pn} 
(with the condition Y^LiPi — 1) of each state (p, 7^ for all i). Then the knowl- 
edge of the underlying physical laws at this scale is incorporated into a probability 
distribution for the accessible states. It is possible to find a quantity measuring the 
amount of " information" . Under to the most elementary conditions of consistency, 
Shannon |§ determined the unique function H(pi,p2, ■■■,Pn) that accounts for the 
"information" stored in a system: 



where K is a positive constant. The quantity H is called information. In the case 
of a crystal, a state x c would be the most probable p c ~ 1, and all others Xi would 
be very improbable, pi ~ i 7^ c. Then H c ~ 0. On the other side, equiprobability 
characterizes an isolated ideal gas, pi ~ 1/N so H g ~ K log N, i.e., the maximum 
of information for a N-system. (Notice that if one assumes equiprobability and 
K — k = Boltzmann constant, H is identified with the thermodinamic entropy 
(S = ftlogiV")). Any other N-system will have an amount of information between 
those two extrema. 

Let us propose a definition of disequilibrium D |] in a N-system. The intuitive 
notion suggests that some kind of distance from an equiprobable distribution should 
be adopted. Two requirements are imposed on the magnitude of D: D > in order 
to have a positive measure of " complexity" and D = on the limit of equiprobability. 
The straightforward solution is to add the quadratic distances of each state to the 
equiprobability as follows: 



N 



H = -KY^Pi^ogPi 



(1) 



i=l 




(2) 
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According to this definition, a crystal has maximum disequilibrium (for the dominant 
state, p c ~ 1, and D c — > 1 for N ^ oo) while the disequilibrium for an ideal gas 
vanishes (D g ~ 0) by construction. For any other system D will have a value 
between these two extrema. 

We now introduce the definition of complexity C of a iV-system. This is simply 
the interplay between the information stored in the system and its disequilibrium: 

C = H-D = - |>logp*) • (X> - ^) 2 ) (3) 

This definition fits the intuitive arguments. For a crystal, disequilibrium is large 
but the information stored is vanishingly small, so C ~ 0. On the other hand, H is 
large for an ideal gas, but D is small, so C ~ as well. Any other system will have 
an intermediate behavior and therefore C > 0. 

As was intuitively suggested, the definition of complexity (3) also depends on 
the scale. At each scale of observation a new set of accessible states appears with its 
corresponding probability distribution so that complexity changes. Physical laws at 
each level of observation allow us to infer the probability distribution of the new set 
of accessible states, and therefore different values for H, D and C will be obtained. 
(In the most complicated situations, where there exist extremely many different 
states, there are methods to calculate functions of the probability distribution |J). 
The passage to the case of a continuum number of states, x, is straighforward. 
Thus we must treat with probability distributions with a continuum support, p(x), 
and normalization condition f_™ p(x)dx = 1. Disequilibrium has the limit D = 
p 2 {x)dx and the complexity is defined by: 



/ r+oo 

C = H ■ D = — (K / p(x) logp(x)dx 




Direct simulations of the definition give the values of C for general N-systems. 
The set of all the possible distributions {p±,P2, ■■■,Pn} where an N-system could be 
found is sampled. For the sake of simplicity H is normalized to the interval [0, 1]. 
This magnitude is called H. Thus H = J2i=i Pi l°gPi/ log N. For each distribution 
{pi} the normalized information H({pi}), and the disequilibrium D({pi}) (eq. 2) are 
calculated. In each case the normalized complexity C = H ■ D is obtained and the 
pair (H,C) stored. These two magnitudes are plotted on a diagram (H,C(H)) in 
order to verify the qualitative behavior predicted in figure 1. For N=2 an analytical 
expression for the curve C(H) is obtained. If the probability of one state is p\ = x, 
that of the second one is simply p2 = 1 — x. The complexity of the system will be: 

C(x) = H(x) ■ D(x) = -j^2 [* log (^) + log(l - x)} ■ 2(x - \) 2 (5) 
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Complexity as a function of H is shown in figure 2a. It vanishes for the two simplest 
2-systems: the crystal (H — 0; p\ — 1, P2 = 0) and the ideal gas (H — 1; p\ — 1/2, 
P2 = 1/2). Let us notice that this curve is the simplest one that fulfills all the 
conditions discussed in the introduction. The largest complexity is reached for 
~H ~ 1/2 and its value is: C(x ~ 0.11) ~ (7-log2 ~ 0.105. For N>2 the relationship 
between H and C is not univocal anymore. Many different distributions {pi} store 
the same information H but have different complexity C. Figure 2b. displays such 
a behavior for N = 3. If we take the maximum complexity C max (H) associated with 
each H a curve similar to the one shown in a 2-system is recovered. Every 3-system 
will have a complexity below this line. In figure 2a. curves C max (H) for the cases 
N = 3, 5, 7 are also shown. Let us observe the shift of the complexity-curve peak to 
smaller values of entropy for rising N. This fact agrees with the intuition telling us 
that the biggest complexity (number of possibilities of 'complexification') be reached 
for lesser entropies for the systems with bigger number of states. 
Also it is possible to compute these quantities in other relevant physical situations 
and in continuum systems. 

We can now go one a step further. The most important point is that the new 
definition should work in systems out of equilibrium. We use two examples of such 
systems where it is known that very complex dynamics could show up. They are 
the logistic map and the 'Lorenz' map. 

Logistic Map: The mapping x n+ i = a ■ x n (l — x n ), a G [0,4], is a well known 
chaotic system. There are two points in this system where the behaviour is extremely 
"complex". The first is the accumulation point of the subharmonic cascade. The 
second is the transition point from chaos to a period three orbit via intermittency 
U . We are going to discuss and study the complexity in the second case only, which 
corresponds to parameter values around a t ~ 3.8284. Complexity must increase 
as (a — a t ) — * when a < a t because the closer we get to the critical point 
a t the more improbable and unpredictable is the firing and development of the 
intermittent bursts. On the contrary, when a > a t the system becomes periodic 
and does not have any complexity. Complexity M undergoes a rapid increase in 
the intermittency region and a sharp transition to zero at the transition point (Fig. 
3a). (This resembles the curve of the specific heat as a function of temperature in a 
second order phase transition). 

Lorenz Map: Let us consider a mapping that mimics the behavior and development 
of the Lorenz attractor. This is the simplest mapping that includes the main features 
of the first return map of this attractor: {x n+ i = a ■ x n if x n < 0.5 and x n+ i = 
a ■ (x n — 1) + 1 if x n > 0.5} where a G (0, 2). Its dynamic evolution displays three 
different behaviors: 1) R\\ a G (0,1). The system goes to a fixed point (0 or 1) 
and the output is a constant signal; 2) R 2 : a G (1,2). A variable chaotic attractor 
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is present in all this region and a chaotic signal is obtained; 3) R3 : a ~ 2. The 
Bernouilli-shift is reached and the output signal is random. Results for complexity 
are given in figure 3b. In region Ri, C vanishes because there is nothing to explain 
a constant signal. In region R 2 , C 7^ and shows a complicated dependence on 
a. Variations in C are due to changes in the structure of the underlying chaotic 
attractor. When a — > 2 (region R3) a random system is reached, and again C = 0. 

Let us return to the point at which we started the intitial discussion. Any notion 
of complexity in physics fl], || should only be made on the basis of a well defined 
or operational magnitude. But two additional requirements are needed in order to 
obtain a good definition of complexity in physics: (cl) the new magnitude must be 
measurable in many different physical systems and (c2) a comparative relationship 
and a physical interpretation between any two measurements should be possible. 



Many different definitions of complexity have been proposed to date, mainly in 
the realm of computational sciences. Among these, several can be cited: algorithmic 
complexity (Kolmogorov-Chaitin) || ||, the Lempel-Ziv complexity [10], the logical 
depth of Bennett [JTTj], the effective measure complexity of Grassberger |T2|, the 
complexity of a system based in its diversity fl3|], the thermo dynamical depth fl4|l , 
complexities of formal grammars, etc. The definition of complexity proposed in 
this work offers a new point of view, based on a statistical description of systems 
at a given scale. In this scheme the knowledge of the physical laws governing the 
dynamic evolution in that scale is used to find its accessible states and its probability 
distribution. This process would inmediately indicate the value of complexity. In 
essence this is nothing but an interplay between the information stored by the system 
and the distance from equipartition (measure of a probabilistic hierarchy between the 
observed parts) of the probability distribution of its accessible states . Besides giving 
the main features of a " intuitive" notion of complexity, we showed that it sucessfully 
enables us to discern situations regarded as complex, both for a local transition (Fig. 
3a.) and for a global behavior (Fig. 3b.) in systems out of equilibrium. 
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Figure Captions 



Fig 1. Sketch of the intuitive notion of the magnitudes of "information" (H) and 
"disequilibrium" (D) for the physical systems and the behavior intuitively required 
for the magnitude "complexity". The quantity C = H ■ D is proposed to measure 
such a magnitude. 

Fig 2a. Complexity (C = H ■ D) as a function of the information (if) for a 
system with two accessible states (N = 2). Also curves of maximum complexity 
(Cmax) are shown for the cases: N = 3,5, 7. 

Fig 2b. In general, dependence of complexity (C) on information (if) is not 
univocal: many distributions {pi} can present the same value of if but different C. 
This is shown in the case N = 3. 

Fig 3a. Behavior of complexity C in the transition point (a t ~ 3.8284) where 
the system (logistic map) goes from a chaotic dynamics (a < a t ) to a period three 
orbit (a > a t ) by intermittency. 

Fig 3b. Complexity displayed by the 'Lorenz' map for a region of the parameter 
space a G [0,2]. (See explanation in the text). (Calculations have been done for a 
scale n — 12, n = length of the binary strings analyzed). 
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